64

6

The Nature of Information

6.3

Accuracy, Meaning, and Effect

6.3.1

Accuracy

In the preceding sections, we have focused on the information gained when a certain

signal, or sequence of signals, is received. The quantity of this informationupper II has been

formalized according to its statistical properties. upper II is of particular relevance when

considering how accurately a certain sequence of symbols can be transmitted. This

question will be considered in more detail in Chap. 7. For now, let us merely note

that no physical device can discriminate between pieces of information differing by

arbitrarily small amounts. In the case of a photographic detector, for example, dimin-

ishing the difference will require larger and larger detectors in order to discriminate,

but photon noise places an ultimate limitation in the way of achieving arbitrarily

small detection.

A communication system depending on setting the position of a pointer on a dial

to 1 of 6000 positions and letting the position be observed by the distant recipient of

the message through a telescope, while allowing a comfortably large range of signs

to be transmitted, would be hopelessly prone to reading errors, and it was long ago

realized that far more reliable communication could be achieved by using a small

number of unambiguously uninterpretable signs (e.g., signalling flags at sea) that

could be combined to generate complex messages. ²⁴

Practical information space is thus normally discrete; for example, meteorological

bulletins do not generally give the actual wind speed in kilometres per hour and the

direction in degrees, but refer to 1 of the 13 points of the Beaufort scale and 1 of the

8 compass points. The information space is therefore a finite 2-space with 8 times 138 × 13

elements.

The rule for determining the distance between two words (i.e., the metric of

information space) is most conveniently perceived if the words are encoded in binary

form. The Hamming distance is the number of digit places in which the two words

differ. ²⁵ This metric satisfies the usual rules for distance; that is, if aa, bb, and cc are

three points in the space and upper D left parenthesis a comma b right parenthesisD(a, b) is the distance between aa and bb, then

24 The same principle applies, in vastly extended form, to the principal systems of writing extant

on Earth. In the Chinese system one character, which may be quite elaborate, represents an entire

word, which could itself represent (often in a context-dependent fashion) an entire concept. In the

alphabetical system, words are built up from syllables. Where there is no difficulty in perceiving a

text in full detail, preferably a whole page at a time, the Chinese system must be superior, having

more force of expression and enabling the information to be appraised more rapidly. In other cases,

such as transmitting messages long distances through a noisy channel, the alphabetic system has

evident advantages.

25 Cf. J.E. Surrick and L.M.Conant, Laddergrams, Sears, New York (1927): “Turn bell into ring in

six moves” and so forth; and Sect. 17.4.3.